- Overview of probability
- Probability
- Random variables
- The Central limit theorem
- Simulation
Overview of probability. Simulation
Marc Comas-Cufí
Euler diagram for three events A, B and C
Given two events \(A\) and \(B\).
What are probabilities?
Frequentist. Probabilities are long run relative frequencies of events.
Bayesian. Probabilities are used to quantify our uncertainty about events.
Example:
The conditional probability of one event \(A\) given another event \(B\) is defined as
\[ \text{P}(A|B) = \frac{\text{P}(A\cap B)}{\text{P}(B)}. \]
Bayes’ rule
For two events \(A\) and \(B\). Then,
\[ \begin{array} \text{P}(A|B) &=& \frac{\text{P}(A) \text{P}(B|A)}{\text{P}(B)}\\ &=& \frac{\text{P}(A) \text{P}(B|A)}{\text{P}(A) \text{P}(B|A) + \text{P}(A^c) \text{P}(B|A^c)}. \end{array} \]
sample().sample() picks elements without replacement. To force replacement:breast_cancer = c('yes', 'no')
sample(breast_cancer, 200, replace = TRUE, prob = c(0.004, 0.996))
#> [1] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [13] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [25] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [37] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [49] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [61] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [73] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [85] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [97] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [109] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [121] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [133] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [145] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [157] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [169] "no" "no" "no" "no" "no" "no" "no" "no" "yes" "no" "no" "no"
#> [181] "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no" "no"
#> [193] "no" "no" "no" "no" "no" "no" "no" "no"Suppose you are a women in your 40s, and you decide to have a medical test for breast cancer called a mammogram.
Information:
library(tidyverse)
N = 100000
test_result_sampling = function(breast_cancer){
if(breast_cancer == 'yes'){
sample(c('+', '-'), 1, prob = c(0.8, 0.2))
}else{
sample(c('+', '-'), 1, prob = c(0.1, 0.9))
}
}
women40s = tibble(
breast_cancer = sample(c('yes', 'no'), N, replace = TRUE, prob = c(0.004, 0.996)),
test_result = map_chr(breast_cancer, ~test_result_sampling(.x))
)Use probability theory to calculate the exact probability of a women having breast cancer once we know she got a positive test.
Hint. Consider the following events:
\[ \text{P}(A) = \text{P}(A|B). \]
\[ \text{P}(A \cap B) = \text{P}(A) \text{P}(B). \]
\[ \text{P}(A|C) = \text{P}(A|B \cap C), \]
or
\[ \text{P}(A \cap B | C) = \text{P}(A |C)\text{P}(B |C). \]
Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?
The expected value of a r.v. \(X\), denoted \(\mathbb{E}[X]\), is the mean value we expect after an infinite number of runs.
For a discrete r.v. \(X\), the expected value is \[ \mathbb{E}[X] = \sum_{x} x \;p(x) \;\;\;\;\;\;\;\;\;\;[\sum_{x} x \;\text{P}(X = x)] \]
For a continuous r.v. \(X\), the expected value is \[ \mathbb{E}[X] = \int_{-\infty}^{\infty} x \;f(x) dx \]
Example: the expected value of the r.v. with the value of rolling a 6-faced dice is
\[ 1 \frac{1}{6} + 2 \frac{1}{6} + 3 \frac{1}{6} + 4 \frac{1}{6} + 5 \frac{1}{6} + 6 \frac{1}{6} = 3.5 \]
\[ \text{var}[X] = \mathbb{E}[ \,(X - \mathbb{E}[X])^2\, ]. \]
\[ \text{cov}[X,Y] = \mathbb{E}[ \,(X - \mathbb{E}[X]) (Y - \mathbb{E}[Y]) \, ]. \]
The categorical distribution is the distribution obtained after associating natural number \(1\) to \(k\) to a set of events with a certain probability \(\boldsymbol\pi = (\pi_1, \dots, \pi_k)\).
The gaussian (normal) distribution is the distribution that reduces the probability exponentially (with velocity \(1/\sigma^2\)) for events far from a certain center (\(\mu\)).
The binomial distribution counts the number of success after repeating a certain experiments \(n\) times when the probability of success is \(\pi\).
The Poisson distribution counts the number of success detected in an interval of time or space when the expected number of successes is \(\lambda\).
Thing about possible variables that can be modelled with one of the four seen distributions:
Think about possible parameters to use.
The joint probability distribution is a multivariate model for random variables. The joint probability distribution of r.v. \(X_1, \dots, X_k\) is completely determined by providing
\[ p(x_1,\dots,x_k) = \text{P}(X_1 = x_1 \cap \cdots \cap X_k = x_k), \]
\[ f(x_1,\dots,x_k). \]
Example: The log-normal distribution is defined as the distribution obtained after exponentiating a gaussian r.v., \(f(x) = e^x\), \(Y = e^X\), where \(X \sim N(\mu,\sigma)\). Check using simulating gaussian r.v. that
If \(X_1, \dots, X_n\) are r.v. equally distributed with expected value \(\mathbb{E}[X]\) and variance \(\text{var}[X]\). Then, when \(n\) is big, we have
\[ \bar{X} = \frac{X_1 + \cdots + X_n}{n} \sim N(\mathbb{E}[X], \sqrt{\text{var}[X]/n}) \]
Next plot shows the distribution of 1000 realisations of a r.v. obtained as the average of 30 binomial distributions with parameters \(n=10\) and \(\pi=0.25\). Blue line represents the gaussian distribution with \(\mu=n \pi\) and \(\sigma^2=\sqrt{n \pi (1-\pi) / 30}\)
If \(X_1, \dots, X_n\) are r.v. equally distributed with expected value \(\mathbb{E}[X]\) and variance \(\text{var}[X]\). Then, when \(n\) is big, we have
\[ \frac{\bar{X} - \mathbb{E}[X]}{\sqrt{\text{var}[X]/n}} \sim N(0, 1) \]
Next plot shows the distribution of 1000 standardised realisations of a r.v. obtained as the average of 30 binomial distributions with parameters \(n=10\) and \(\pi=0.25\). Blue line represents the gaussian distribution with \(\mu=0\) and \(\sigma^2=1\)
If \(x_1, \dots, x_n\) is a sample i.i.d with expected value \(\mathbb{E}[X]\). Then, when \(n\) is big, we have
\[ \frac{\bar{x} - \mathbb{E}[X]}{\sqrt{s_x/n}} \sim t_{n-1} \]
Next plot shows the distribution of 1000 standardised realisations of a r.v. obtained as the average of 30 binomial distributions with parameters \(n=10\) and \(\pi=0.25\). Normal and Student distributionss are compared.